Quote Clustering in Online News

The notion that information moves through social networks has been widely discussed[3], however, with the growing availability of large digital corpora, the ability to quantitatively model this phenomenon is new. To this end we explore a large corpus of online news quotations looking for cases of noisy reproduction and the factors which influence such noise. An essential step in this process is distinguishing mutational variants from entirely independent but similar quotations. Given that the question of whether two quotes really were derived from the same original utterance cannot be known with complete certainty, it is not immediately apparent how to make progress on such a task. Our appraoch is twofold: On the one hand we attempt to frame the problem in terms of supervised learning, annotating data using Mechanical Turk. On the other hand, we approach the problem more from the perspective of unsupervised clustering, projecting the data into a variety of metrics which allows us to test and extend our linguistic intuitions about the dataset.